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Abstract — In this paper, we investigate the additive Gaussian 
noise channel with noisy feedback. We consider the setup 
of linear coding of the feedback information and Gaussian 
signaling of the message (i.e. Cover-Pombra Scheme). Then, we 
derive the upper and lower bounds on the largest achievable 
rate for this setup. We show that these two bounds can be 
obtained by solving two convex optimization problems. Finally, 
we present some simulations and discussion. 

I. Introduction 

The study of the additive Gaussian noise channel with 
feedback has been a hot research topic for decades. So far, 
a large body of work has looked at the ideal feedback case 
and obtained many notable results [l]-[5]. As an illustration, 
it is known that noiseless feedback improves the error 
exponent and reduces the coding complexity. However, only 
few papers have studied channels with noisy feedback and 
many open problems still exist. Literature on noisy feedback 
problems can be largely classified into two categories. The 
first category studies the usefulness of noisy feedback by 
investigating reliability functions and error exponents [6], 
[7]. The second focuses on the derivation of coding schemes 
based on the well-known Schalkwijk-Kailath scheme. We 
refer interested readers to [2], [8]— [ 11] for details. 

In this paper, we investigate the behavior of the largest 
achievable rate of the additive Gaussian noise channel with 
noisy feedback, under the restriction of linear feedback 
coding scheme (i.e. Cover-Pombra Scheme). We derive in- 
formative upper and lower bounds on the largest achievable 
rate. This upper bound outperforms the bound presented 
in [12], especially, in the case of having small feedback 
noise. The lower bound shows the enhancement, in terms 
of the achievable rate, by exploiting the noisy feedback link. 
Additionally, the derived bounds provide insight on how the 
noisy feedback channel behaves with respect to the feedback 
noise. 

The paper is organized as follows. In Section HU we 
introduce some important definitions and lemmas, which are 
used throughout the paper. In Section III, we introduce the 
signal model of the noisy feedback channel and then derive 
the formula of the achievable rate. In Section [TV] and [V] we 
derive an upper bound and a lower bound on the largest 
achievable rate, respectively. We present some simulation 
results and discussion in Section [VI] and conclude the paper 
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in Section IVIII 

Notations: Uppercase and corresponding lowercase letters 
(e.g.Y,Z,y,z) denotes random variables and realizations, 
respectively, x" represents the vector [xi,X2,- ■ ■ ,x„] T and 
x° = 0. I„ represents an n x n identity matrix. K„ > (K„ > 0) 
denotes that the n x n matrix K„ is positive definite (semi- 
definite), log denotes the logarithm base 2 and OlogO = 0. 
The expectation operator over X is presented as E(X). 

II. Technical Preliminaries 

In this section, we review some main definitions and 
Lemmas in information theory. 

Definition 1: [13] The mutual information I(X;Y) be- 
tween two random variables with joint density f(x,y) is 
defined as 



I(X;Y) 
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Let h(X) denote the differential entropy of a random 
variable X. Then it is clear that 

I(X;Y)=h(Y)-h(Y\X) 

Next, we present a useful Lemma as follows. 

Lemma 1: [13] Let the random vector Xel" have zero 
mean and covariance K v ,„ = EXX T (i.e. Ky = KX/Xj, 1 < 
i,j < n). Then 

h(X) < -log(2n:e) n detK^„ 

with equality if and only if X ~ N(Q,K XJ1 ). 

Finally, we introduce an meaningful notion of directivity 
to the information flow through a channel [3]. 

Definition 2: The directed information I(X" — > Y") from 
a sequence X" to a sequence Y" is defined by 

I{X n ^Y") = Y^I{X i \Y i \Y i - x ). 

i=\ 

It has been shown in [3], when feedback is present, 
directed information is a more useful quantity than the 
traditional mutual information. 

III. Modeling and Achievable Rate 
A. Modeling 

Consider a point-to-point additive Gaussian noise channel 
with access to an additive Gaussian noise feedback link. 
Since it is difficult to characterize the capacity in general, 



x" = s" + B„(w" + v") 
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Fig. 1. A additive Gaussian noise channel with noisy feedback 



we consider a system with linear encoding of the feedback 
signal and Gaussian signaling of the message (i.e. shown in 
a vector form in Fig. Q~|i [12]. 

The channel input signal: x" = s" + B„ (w" + v") 

The channel output signal: y" = s" +B„(w" + v") + w" 

The power constraint: fr(K 4 -,„ +B„(K vl ,,„ +K V ,„)B^) < nP 

where w" ~ A^(0, K w .„) and v" ~Af(0,K,,„). We assume 
that K w .„ > and K,,,„ > and, therefore, these covariance 
matrices are invertible. Here, s" ^ N(0,K SJ1 ) is the message 
information vector. B„ is an n x « strictly lower triangular 
linear encoding matrix. Note that the one-step delay in the 
feedback link is dealt with the structure of matrix B„ and 
random variables s",v",w" are automatically assumed to be 
independent. 

Remark 1: The bounds provided later in the paper are 
only valid for this specific setup. In other words, the 
bounds may not be true for the capacity (i.e. the maximum 
achievable rate over all encoding strategies). Informative 
computable bounds on the capacity of channels with noisy 
feedback, however, are not known. 

B. n-block Achievable Rate 

Based on the above model, we obtain the n-block achiev- 
able rate R n ° lsy as 

R n ° is y =-I(M;Y n ) 
n 

=-I(S n ;Y n ) (a) 
n 

1 -{h(Y")-h(Y n \S n )) 



n 
1 

-log 



1 de t((I„ +B„)K H ,„(I„ +B„) T + B„K V ,X +K,,„) 
det ((I„ + B„)K VV ,„ (I„ +B n ) T + B„K V ,„BJ) 



(a) follows the fact that the message information vector S n is 
determined by the message M and the last equality follows 
from Lemma Q] We denote the largest n-block achievable 
rate under power constraint P as R„''ma X (P) where 



If the feedback is ideal (i.e. K v „ = 0„), expression (Q~|i is 
simplified to 

nfb .. 1 det((I„+B„)K H ,„(I„+B„) r +K,,„) 

c n (P)= max -log -— — '— 

b„.k,„« & detK Wi „ 

where C„ (P) denotes the n-block capacity of the additive 
Gaussian noise channel with ideal feedback and fr(K s ,„ + 
B„K VV ,„B^) < nP. Note that the fact det(I„ + B„) = det(I„ + 
B„) T = 1 is applied in the simplification. [14] found that 
this problem can be transformed into a well-known convex 
optimization problem called the matrix determinant maxi- 
mization (max-det) problem and, then, solved efficiently. 

If there exists no feedback (i.e. B„ = 0„), expression (QJ 
is simplified to 



(i) 



" v ' K.„, n b detK„ 



where Cn Pen (P) denotes the n-block capacity of the open- 
loop additive Gaussian noise channel and fr(K s „) < nP. It 
is well known that this optimization problem can be solved 
by water-filling on the eigenvalues of K^„. 

Unlike the above two simplified forms, the optimization 
problem (fl3 is difficult to solve (i.e. not easily expressed 
in a convex form). Furthermore, the expression for R™'" 
does not provide much information about the behavior of the 
channel with respect to the feedback noise. This motivates us 
to derive the effective upper and lower bounds On R n max(P\ 
from which we can discover some characterizations of the 
noisy feedback channel. 

IV. An Upper Bound on R n n °}Zc{P) 

In this section, we first present an upper bound on the 
achievable rate of a general channel with additive noise 
feedback (i.e. not restricted to be an additive Gaussian noise 
channel). Then, we show that, for the noisy feedback Gaus- 
sian channel under linear feedback coding scheme, this upper 
bound can be obtained by solving a convex optimization 
problem. 

Lemma 2: For a point-to-point communication channel 
with additive noise feedback, we have 

7(M;F") < I(X" -> Y"\V") < I(X" -> Y") < I(X n ;Y n ) 

The first and second equalities hold if there exists an ideal 
feedback (i.e. V" = 0). The last equality holds if these exists 
no feedback. 
Proof: 

I(M;Y") 
=h(M)-h{M\Y n ) 
<h(M)-h(M\Y n ,V) 

{ ^h(M\V")-h{M\Y",V n ) 

=I(M;Y"\V") 

=h(Y"\V")-h(Y"\M 7 V") 



£ h (F \y'- 1 , V " ) - h (Y t \ r~ 1 , M, V " ) 



convex optimization problem. 



1=1 



(b) 



= £ h(Yj\Y'~\ V) - A(Fi|F i_1 ,X i ,y' 1 ) 
i=i 

= £/(r;F,|F'-\n 
i=i 

=/(x n H.F"|y") 

(a) follows from the fact that M and V" are independent, (b) 
follows from the fact that X' can be determined by M and the 
outputs of the feedback link (i.e. Y'^ 1 + V'~ l ). (c) follows 
from the Markov chain M -> (F' -1 ,^', V") -4 F. Note that 
the equality holds if V" = 0. 
Next, we have 



«/). 



£ a (f | r 1 , v " ) - /j(f | y f ~ 1 , x> , v " ) 



i=i 



£/z(F(F , '- 1 ,y")-/ 2 (F|F , '- 1 ,r) 



i=i 



<£/z(FiF , '- 1 )-ft(FiF , '- 1 ,x'') 



i=i 

= £/(r';F|F i '- 1 ) 

i=i 

where (d) follows from step (c) and (e) follows from the 
Markov chain V" -» (F i-1 ,X') -> F. Note that the equality 
holds if V" = 0. 

The last inequality I(X" -> F") < 7(X";F") is proved in 
[3]. ■ 

It is known that, for ideal feedback channels, the directed 
information I(X n — > F") is an appropriate measure on the 
achievable rate and, therefore, can correctly characterize the 
ideal feedback channel capacity [15]. However, Lemma [2] 
shows that, for noisy feedback channels, the conditional 
directed information I(X" — > Y"\V") performs as a better 
upper bound on the achievable rate than I(X" — > Y"). This 
motivates us to take I(X n — > Y"\V") as an upper bound on 
C^ olsy (P) instead of I(X n — > Y") and investigate the following 
optimization problem. 

maximize -I(X" -> F" I V" ) 
b„ ,k,.„ n 

subject to tr(K x .„)<nP, K,,„ > 

B„ is strictly lower triangular 



(2) 



Next, we show that the above optimization problem can 
be transformed into a convex form. 

Theorem 1: An upper bound on the largest n-block 
achievable rate of linear feedback coding scheme for Gaus- 
sian channels with additive noise feedback, as shown in Fig. 
Q] can be obtained as the optimal value of the following 



maximize — log det 
h„,b„ 2n 



B„ H„ 



1 

27i 



logdet(K- I 1 K llV7 ) 



subject to t r (H„ - K w<n W n - B„K H ,„ - K H ,„ ) < nP 
H„ I„ + Bjf B^ 
In + B„ K^ n 0„ 



K v,i!. 



>0 



B„ 0„ 

B„ is strictly lower triangular 



Proof: We are beginning with the optimization 
problem ©. Let H„ = (I„ + B„)K H ,„(I„ + B„) r + K Si „ + 
B„K V)n B^, we have 

I(X"^Y"\V") 

( = ) £/z(F|F i - 1 ,n-/ J (F|F i - 1 ,x ! ') 

i=l 

= £ h{Yi\Y'-\ V") - h(X, + Wi\Y l -\X\ W 1 ' 1 ) 
i=i 

= f j h(Y i \Y-\V)-h(W,\W i - 1 ) 
;=i 

=h(Y"\V")-h(W") 

=h{% + B n )W" + B„V" + S n \V n ) - h(W") 
=h((l„+V„)W n + S")-h(W n ) 
(6}l lo det((I„ +B n )K H , n (I„ +B,,) r + K„„) 
2 detK Wj „ 

= l Jo det(H„~B„K t ,,X) 
2 detK Wi „ 

where (a) follows from step (e) in the proof of Lemma|2]and 
(b) follows from Lemma Q] 
Also, we have 

tr{K x , n ) < nP <s> tr(K s ,„ +B„(K V ,„ +K w ,„)Bj) < nP 

& rr(H„ - K W ,X - B n K -n - K v ,,„) < nP 

Next, we have the following equivalences by applying the 
Schur complement. 

= det(H, 7 -B„K u X)detK-,!. 



(1) .det 

(2) . 



v.n 

B„ H„ 



Kj jK > H„ — (I„ + B„)K U .„(I„ +B„) r — B„K,,„B^ >0 



H„ I„ + B„ B^ 
i + B„ K^* 0„ 
B„ 0„ K v i 



>0 



By taking simple replacements on the optimization prob- 
lem (O, the proof is complete. ■ 

Note that H„ is the covariance of the received signal 
y n . This upper bound provides interesting insight because 
it shows that the effect of the noise in the feedback link can 
be formulated as the allocation of the channel input power 
P. As shown in the proof of Theorem Q] we can rewrite (f2]i 



x" = s"+B„(w" + v") 
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Fig. 2. A modified additive Gaussian noise channel with noisy feedback 



as 



1, det((I„+B„)K lr ,„(I„ 

maximize — log 

B„,K 5 ,„ 2n detK VV; „ 

subject to fr(K ii „+B„(K Vi „+K Wi „)Bj) < nP, K,,„ > 

B„ is strictly lower triangular 

(3) 

K v „ herein only affects the power constraint. If K v ,„ = 0„, 
the optimization problem ([3]) recovers the n-block capacity 
of channels with ideal feedback [4]. This implies that, for 
channels with noisy feedback, it is necessary to assign certain 
amount of power to cancel the effect of the feedback noise 
such that the message can be recovered by the decoder with 
an arbitrarily small error probability. If the noise in the 
feedback link increases (i.e. K v ,„ grows large in some sense), 
the feedback benefit in increasing reliable transmission rate 
vanishes. Namely, the noisy feedback system behaves like a 
nonfeedback system since, due to the power constraint, B„ 
approaches 0„ as K v .„ grows. Note that this upper bound is 
tight when K v „ is either small or large. 

V. A Lower Bound on R"°,nL(P) 

First of all, we consider a new channel with noisy feed- 
back, as shown in Figj2j An identical Gaussian noise v is 
added on the channel output. Then, 

y"=x"+w" + v" 

=s" + (B„+I„)(w"+v") 

Since the decoder is not able to access the new additive 
noise, the largest achievable rate of the new channel must 
be a lower bound on that of the original channel (shown in 
FiglD- This motivates us to solve the following optimization 
problem for obtaining this lower bound. 



maximize 

B/i ,Kci, 



-I(M;f n ) 
n 



subject to tr(K Xt „) < nP, K ( ,„ > 

B„ is strictly lower triangular 



(4) 



Similarly, we show that the above optimization problem 
can be transformed into a convex form. 

Theorem 2: A lower bound on the largest n-block achiev- 
able rate of linear feedback coding scheme for Gaussian 
channels with additive noise feedback, as shown in FigQ] 



2 
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Fig. 3. Comparison of upper bounds on C% msy of the lst-MV channel 

can be obtained as the optimal value of the following convex 
optimization problem. 

-!- logdetH„ - log detK vn ,„ 
In In 

t r(H„ — K«iv.nB„ — B„K V1 



maximize 

H„ ,B„ 

subject to 



K WVi „) < nP 



H„ 

I« + B„ 



K 



-l 



>0 



B„ is strictly lower triangular 

where K v „.„ = K v ,„ + K w>n . 

The proof is similar to that in [14] by considering the 
above setup. 

Remark 2: This lower bound is tight when K,,„ = and 
becomes increasing loose as K v „ increases. This lower 
bound becomes useless when it is below the corresponding 
nonfeedback capacity. Since we restrict the feedback coding 
scheme to be linear, R"n.max is in fact a lower bound of the 
capacity. Therefore, the lower bound of R"„".max is obviously 
valid for the capacity. 

VI. Simulations and Discussion 

In this section, we performed simulations for a specific 
channel with noisy feedback link. We assumed that the 
forward channel is created by a first order moving average 
(lst-MV) Gaussian process. Namely, 

Wi = U i + aU i - l 

where Uj is a white Gaussian process with zero mean and 
unit variance. We also assumed that the feedback link is 
created by an additive white Gaussian noise with K, y , = al„ 
(a > 0). Because of the practical computation limit, we take 
coding block length n = 30 and power limit P = 10. 

We first compared our upper bound (i.e. Theorem [TJ) with 
the one presented in [12] (Lemma 3). See Figj3] As a in- 
creases, both of the upper bounds approach the nonfeedback 
capacity, which implies the "shut off" of the feedback link. 
However, our bound is much more tight, especially, in the 
small feedback noise region. Note that, when the feedback 
noise vanishes, the bound in [12] on any noisy feedback 
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Fig. 4. The bounds on C n msy of the lst-MV channel with a = 0.1 
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Fig. 5. The bounds on C%° ,sy of the lst-MV channel with a = 0.5 



channel grows to infinity and, thus, should be truncated by 
the ideal feedback capacity. In contrast, our bound converges 
to the ideal feedback capacity in this case. Therefore, we may 
claim that our upper bound is better in general. 

Next, we computed the bounds derived in our paper for 
averaging statistic a = 0.3,0.5,0.9 in the lst-MV channel, as 
shown in Fig. [4]|6] Generally, the plots show that the largest 
achievable rate R'n'.max, which is in the region between the 
upper and lower bounds, sharply decreases as a grows. When 
a grows large enough (i.e. a = 0.8 in FigHJi, the feedback 
rate-increasing enhancement almost shuts off and, thus, the 
feedback system behaves like a nonfeedback system. Based 
on this observation, we conclude that the achievable rate of 
the additive Gaussian noise channel with noisy feedback is 
sensitive to the feedback noise under the linear feedback 
scheme. 

Additionally, the plots show that the decrease of the 
achievable rate with o~ is lesser as a grows. This indicates 
that the achievable rate is less sensitive to the feedback 
noise if the channel has more correlated channel noise. This 
intuitively makes sense since utilize a feedback link for 
channels with more correlated channel noise would increase 
more transmission rate and, therefore, the corruption effect 
of the feedback noise is relatively reduced in this case. 
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Fig. 6. The bounds on C„ of the lst-MV channel with a = 0.9 



VII. Conclusion 

We have derived the upper and lower bounds on the largest 
achievable rate for a linear feedback coding setup. It is shown 
that these two bounds can be obtained as the optimal values 
of two convex optimization problems. Furthermore, these 
bounds provide us the following insight: 1. The achievable 
rate is very sensitive to the feedback noise. 2. The achievable 
rate of channels with more correlated channel noise is less 
sensitive to the feedback noise. 
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